Semantic image Segmentation based on Convolutional Neural Networks (CNNs) is one of the main approaches in computer vision area. In convolutional neural network-based approaches, a pre-trained CNN which is trained on the large image classification datasets is generally used as a backend to extract features (image descriptors) from the images. Whereas, the special size of output features from CNN backends are smaller than the input images, by stacking multiple deconvolutional layers to the last layer of backend network, the dimension of output will be the same as the input image. Segmentation using local image descriptors without involving relationships between these local descriptors yield weak and uneven Segmentation results. Inspired by these observations, in this research we propose Context-Aware Features (CAF) unit. CAF unit generate image-level features using local-image descriptors. This unit can be integrated into different Semantic image Segmentation architectures. In this study, by adding the proposed CAF unit to the Fully Convolutional Network (FCN) and DeepLab-v3-plus base architectures, the FCN-CAF and DeepLab-v3-plus-CAF architectures are proposed respectively. PASCAL VOC2012 datasets have been used to train the proposed architectures. Experimental results show that the proposed architectures have 2. 7% and 1. 81% accuracy improvement (mIoU) compared to the related basic architectures, respectively.

In recent years, Convolutional Neural Networks (CNNs) have made significant strides in the field of Segmentation, particularly in Semantic Segmentation where both accuracy and efficiency are crucial. However, despite their high accuracy, these deep networks are not practical for real-time use due to their low inference speed. This issue has prompted researchers to explore various techniques to improve the efficiency of CNNs. One such technique is knowledge distillation, which involves transferring knowledge from a larger, cumbersome (teacher) model to a smaller, more compact (student) model. This paper proposes a simple yet efficient approach to address the issue of low inference speed in CNNs using knowledge distillation. The proposed method involves distilling knowledge from the feature maps of the teacher model to guide the learning of the student model. The approach uses a straightforward technique known as pixel-wise distillation to transfer the feature maps of the last convolution layer of the teacher model to the student model. Additionally, a pair-wise distillation technique is used to transfer pair-wise similarities of the intermediate layers. To validate the effectiveness of the proposed method, extensive experiments were conducted on the PascalVoc 2012 dataset using a state-of-the-art DeepLabV3+ Segmentation network with different backbone architectures. The results showed that the proposed method achieved a balanced mean Intersection over Union (mIoU) and training time.

In this paper, a new approach to weakly supervised Semantic Segmentation is proposed. The main goal in Semantic Segmentation is to assign a Semantic label to each pixel. In weakly supervised setting, each training image is only labeled by the classes they contain, not by their locations. The main contribution of this paper is to simultaneously incorporate the object level and context level information in assigning class label to each pixel of the image. To do this, regions in each image are grouped such that groups of regions in images with the same Semantic label have the same appearance and context. To do this, an iterative move-making algorithm is proposed. At first, each pixel of the image is initially labeled and then model of appearance and context for each class label is learned. Then, Semantic label of each pixel is updated such that the regions with the same sematic label have the same appearance and context in the set of images. In the next step, appearance and context models for each Semantic class are updated. It is repeated until in the two consecutive epochs, labels of the pixels are not changed. To evaluate our proposed approach, it is applied on the MSRC dataset. The obtained results show that our approach outperforms comparable state-of-the-art approaches.

Nowadays, video Semantic Segmentation is used in many applications such as automatic driving, navigation systems, virtual reality systems, etc. In recent years, significant progress has been observed in Semantic Segmentation of images. Since the consecutive frames of a video must be processed with high speed, low latency, and in real time, using Semantic image Segmentation methods on individual video frames is not efficient. Therefore, Semantic Segmentation of video frames in real time and with appropriate accuracy is a challenging topic. In order to encounter the mentioned challenge, a video Semantic Segmentation framework has been introduced. In this method, the previous frames Semantic Segmentation has been used to increase speed and accuracy. For this manner we use the optical flow (change of continuous frames) and a GRU deep neural network called ConvGRU. One of the GRU input is estimation of current frames Semantic Segmentation (resulting from a pre-trained convolutional neural network), and the other one is warping of previous frames Semantic Segmentation along the optical flow. The proposed method has competitive results on accuracy and speed. This method achieves good performances on two challenging video Semantic Segmentation datasets, particularly 83. 1% mIoU on Cityscapes and 79. 8% mIoU on CamVid dataset. Meanwhile, in the proposed method, the Semantic Segmentation speed using a Tesla P4 GPU on the Cityscapes and Camvid datasets has reached 34 and 36. 3 fps, respectively.

Accurate Segmentation of lesions from dermoscopic images is very important for timely diagnosis and treatment of skin cancers. Due to the variety of shapes, sizes, colors, and locations of lesions in dermoscopic images, automatic Segmentation of skin lesions remains a challenge. In this study, a two-stage method for the Segmentation of skin lesions based on deep learning is presented. In the first stage, convolutional neural networks (CNNs) estimate the approximate size and location of the lesion. A sub-image around the estimated bounding box is cropped from the original image. The sub-image is resized to an image of a predefined size. In order to segment the exact area of the lesion from the normal image, other CNNs are used in the DeepLab structure. The accuracy of the normalization stage has a significant impact on the final performance. In order to increase the normalization accuracy, a combination of four networks in the structure of Yolov3 is used. Two approaches are proposed to combine Yolov3 structures. The Segmentation results of two networks in the DeepLab v3+ structure are also combined to improve the performance of the second stage. Another challenge is the small number of training images. To overcome this problem, the data augmentation is used, as well as using different modes of an image in each stage. In order to evaluate the proposed method, experiments are performed on the well-known ISBI 2017 dataset. Experimental results show that the proposed lesion Segmentation method outperforms the state-of-the-art methods.

Recent advancements in Weakly Supervised Semantic Segmentation (WSSS) have highlighted the use of image-level class labels as a form of supervision. Many methods use pseudo-labels from class activation maps (CAMs) to address the limited spatial information in class labels. However, CAMs generated from Convolutional Neural Networks (CNNs) are often led to focus on prominent features, making it difficult to distinguish foreground objects from their backgrounds. While recent studies show that features from Vision Transformers (ViTs) are more effective in capturing the scene layout than CNNs, the use of hierarchical ViTs has not been widely studied in WSSS. This work introduces "SWTformer" and explores the effect of Swin Transformer’s local-to-global view on improving the accuracy of initial seed CAMs. SWTformer-V1 produces CAMs solely based on patch tokens as its input features. SWTformer-V2 enhances this process by integrating a multi-scale feature fusion mechanism and employing a background-aware mechanism that refines the accuracy of localization maps, resulting in better differentiation between objects. Experiments on the Pascal VOC 2012 dataset demonstrate that compared to state-of-the-art models, SWTformer-V1 achieves 0.98% mAP higher in localization accuracy and generates initial localization maps that are 0.82% mIoU higher in accuracy while relying solely on the classification network. SWTformer-V2 enhances the accuracy of the seed CAMs by 5.32% mIoU. Code available at: ttps://github.com/RozhanAhmadi/SWTformer

This paper presents a Semantic method for aerial image Segmentation. Multi-class aerial images are often featured with large intra-class variations and inter-class similarities. Furthermore, shadows, reections and changes in viewpoint, high and varying altitude and variability of natural scene pose serious problems for simultaneous Segmentation. The main purpose of Segmentation of aerial images is to make subsequent recognition phase straightforward. Present algorithm combines two challenging tasks of Segmentation and classification in a manner that no extra recognition phase is needed. This algorithm is supposed to be part of a system which will be developed to automatically locate the appropriate site for Unmanned Aerial Vehicle (UAV) landing. With this perspective, we focused on segregating natural and man-made areas in aerial images. We compared different classifiers and explored the best set of features for this task in an experimental manner. In addition, a certainty based method has been used for integrating color and texture descriptors in a more efficient way. The experimental results over a dataset comprised of 25 high-resolution images show the overall binary Segmentation accuracy rate of 91.34%.

The importance and demand of visual scene understanding have been increasing because of autonomous systems development. Optical flow is known as an important tool for scene understanding. Current optical flow methods present general assumptions and spatial homogeneous for spatial structure of flow. In fact, the optical flow in an image depends on object class and the type of object movement. The first assumption in many methods in this field is the brightness constancy during movements of pixels between frames. This assumption is proven to be inaccurate in general. In this paper, we use recent development of deep convolutional networks in Semantic Segmentation of static scenes to divide an image in to different objects and also depends on type of the object different movement patterns are defined. Next, estimation of the optical flow is performed by using deep neural network for initial image which has been Semantically segmented. The proposed method provides minimum error in optical flow measures for KITTI-2015 database and results in more accurate Segmentation compared to state-of-the-art methods for several natural videos.